Search for Lepton Flavor Violating Decays of the Neutral Kaon 
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The Fermilab KTeV experiment has searched for lepton-flavor- violating decays of the Kl meson 
in three decay modes. We observe no events in the signal region for any of the modes studied, and 
we set the following upper limits for their branching ratios at the 90% CL: BR{Kl — * tt^/i e^) < 



7.56 X 10"'/ BR(Kl -^ 7r"7rV*eT) < 1.64 x W"; BR{tv° -^ /i^e'F) < 3.59 x 10"'". This resuh 
represents a factor of 82 improvement in the branching ratio hmit for Kl -^ vr'^/ie and is the first 
reported limit for Kl — > vr'^Tr^^^e^. 
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PACS numbers: 13.20.Eb, ll.SOFs 



In the Standard Model of particle physics lepton- 
flavor-violating (LEV) decays are possible with non-zero 
neutrino masses and mixing, but the rates for such de- 
cays are far beyond the reach of any current experiment 
[l|. Therefore, the observation of LEV decays would be 
an indication of new physics. Many scenarios for physics 
beyond the Standard Model allow LEV decays. Super- 
symmetry [2|, new massive gauge bosons [1, 3], and Tech- 
nicolor [4|] all can lead to LEV decays which might be 
within reach of current experiments. Searches in K^ de- 
cays are complementary to searches in the charged lepton 
sector, since K^ decays probe the s — > dfie transition 1|. 

In this letter we report on searches for three LEV 
processes in the KTeV experiment at Eermilab. We 
present improved limits on the decays Kj^ — > 7r°/i e^ 
and tt" -^ fi^e"^ (tagged from Kl -^ vr'^Tr'^Tr"), and we 
report the first limit on the decay Kl -^ 7r°7r°/x^e^. 

The KTeV E799-II experiment at Eermilab took data 
in 1997 and 1999. The combined results from both peri- 
ods are presented here. The KTeV beam was produced 
by 800 GeV/c protons from the Tevatron which were di- 
rected onto a BeO target and collimators to create two 
nearly-parallel Kl beams. The beams entered a 65m 
long vacuunr tank which defined the fiducial volume for 
accepted decays. 



Charged particles were detected by two pairs of drift 
chambers separated by an analysis magnet that provided 
a transverse momentum kick of either 0.250 GeV/c (for 
the 1997 data) or 0.150 GeV/c (for the 1999 data). Dis- 
crinrination between charged pions and electrons was pro- 
vided by a set of transition radiation detectors (TRDs) 
behind the last drift chamber. Downstream of the TRDs 
were two planes of trigger hodoscopes, followed by a Csl 
electromagnetic calorimeter, which had an energy resolu- 
tion a{E)/E = 0A5%®2%/^/E{GeV). The calorimeter 
provided powerful electron/pion discrimination based on 
the ratio of energy as measured in the calorimeter (E) to 
momentum as measured in the spectrometer (p) , or E/p. 
The lateral shower shape in the calorimeter provided ad- 
ditional electron/pion discrimination. The Csl calorime- 
ter had two beam holes to allow the undecayed beam 
particles to pass through. A Beam Anti (BA) calorime- 
ter covered the solid angle behind the two beam holes. 
Photon detectors were positioned around the vacuum de- 
cay region, the spectrometer, and the calorimeter to veto 
particles escaping the fiducial region of the detector. 

The muon system was located downstream of the 
calorimeter, shielded by 10 cm of lead followed by 4m of 
steel. Behind the steel was a plane of muon hodoscopes, 
consistiirg of 15cm wide scintillator paddles oriented ver- 



tically. Behind this hodoscope was another meter of steel, 
followed by two more planes of scintillator paddles, one 
oriented vertically and one horizontally. 

The hardware trigger for this analysis required at least 
one hit in the last two banks of muon counters and at 
least three energetic in-time clusters in the Csl calorime- 
ter. The Level 3 software trigger required two tracks 
which formed a good vertex, with one one track having 
an E/p value greater than 0.7, consistent with an elec- 
tron. More detail of the KTeV detector can be found in 

i- 

A detailed Monte Carlo simulation was used to study 

detector performance and acceptance, to simulate back- 
grounds, and to select cuts. For the LFV decays, a uni- 
form phase space decay distribution was assumed. 

The number of K^ decays in our fiducial volume, which 
we refer to as the flux, was determined for each decay 
mode by comparison to a similar decay with a well-known 
branching fraction. Using a normalization mode similar 
to the signal mode cancels many systematic uncertain- 
ties. For the decay Kl -^ 7r°/i^e^, the normalization 



mode was Kl 



TT+TT Tt". 



For Kl -^ n^TT^n^e'^ and 



TT*^ — > fJ, e^ , the normalization mode was Kj^ —> n ;i vi^, 
where tt^ denotes a tt" Dalitz decay, tt" -^ e^e^j. For 
all values of the flux and single event sensitivity quoted 
below, the systematic error was determined by varying 
the analysis cuts and noting the change in the measured 
flux. An additional 2% systematic error on the efficiency 
of the muon trigger was included, since there was no 
muon requirement for either normalization mode. The 
uncertainty in the branching fraction of the normaliza- 
tion modes was included as a systematic error. 

We first consider the decay Kl -^ n^ii^e^ . The signa- 
ture for this decay was two charged tracks (one electron 
and one muon) and two neutral clusters. The charged 
tracks were required to form a good vertex within the 
fiducial decay volume, and both tracks were required to 
match a cluster in the Csl calorimeter. One charged track 
was required to have an E/p ratio within 5% of 1.0 and a 
transverse shower shape consistent with an electromag- 
netic shower. A loose cut on the TRD information (98% 
efficient for electrons) gave an additional cross-check on 
electron identification. The second track was required 
to deposit less than 1 GeV of energy in the calorimeter, 
consistent with a minimum ionizing muon, and to have 
a momentum greater than 8 GeV/c. The projection of 
the downstream segment of the muon track was also re- 
quired to match hits in all three hodoscope planes of the 
muon detector, within a road determined by the expected 
multiple scattering. 

The tt" was reconstructed by its decay to two photons 
which were detected as clusters in the calorimeter with 
no associated charged tracks and with transverse shower 
shapes consistent with an electromagnetic shower. The 
energy and position of the neutral clusters along with 
the location of the charged vertex were used to calculate 



Mj^, the invariant mass of the two photon system. M^^ 
was required to be within 1.4 a of the 7r° mass, where a is 
the 7r° mass resolution of 1.4 MeV/c^, as determined from 
the normalization mode. This requirement was chosen to 
optimize the ratio S/ yB, where S is the number of signal 
events and B is the number of background events. 

The following kinematic cut further reduced back- 
grounds. Assuming a signal mode decay, we calculated 
the square of the 7r° momentum in the Kl rest frame. 
For many backgrounds this quantity has an unphysical 
negative value. We required this quantity to lie between 
and 0.025 (GeV/c)^, where the upper value is the kine- 
matic cutoff in the signal mode. 

The flight direction of the parent K^ can be approxi- 
mated by a line from the center of the target to the decay 
vertex. We defined pt to be the sum of the momentum 
components of all final-state particles perpendicular to 
this direction. For well-reconstructed signal events pi 
should be close to zero. The signal and control regions 
were defined using a likelihood variable L derived from 
pI and A/jrO^ei the invariant mass of the 7r°//e system, 
in the following way. Using signal Monte Carlo, the Kl 
mass distribution was fit with a Gaussian, and the pi 
distribution was fit with a three-component exponential, 
producing probability density functions (PDFs) for these 
variables. Since these variables were found to be un- 
correlated, the joint PDF was defined as the product of 
the two single-variable PDFs. Then L was calculated for 
each event by evaluating the joint PDF at the pi and 
MTfO^e value for that event. The signal (control) region 
was defined by a cut on L chosen to retain 95% (99%) 
of signal Monte Carlo events after all other cuts were 
applied. Both the signal and control regions were blind 
during the analysis. Figure [1] shows the pi — M^o^^ plane 
with Kl -^ 7r"/i*e^ signal Monte Carlo events shown as 
points, and the signal and control regions shown as solid 
contours. 

The dominant background for Kl -^ iT^ir^e^ was the 
decay Kl — » ii^e^Ve (i^es), with a tt decay or pmich 
through to the muon hodoscopes, accompanied by two 
accidental photons faking a tt*^. Since accidental pho- 
tons were often accompanied by other accidental activ- 
ity, we made stringent anti-accidental cuts to reduce this 
background. An event was cut if any additional charged 
tracks were present. We allowed no extra in-time hit pairs 
in the drift chambers upstream of the analysis magnet 
and at most two extra in-time pairs downstream of the 
magnet. We also cut on the number of partial track stubs 
in the upstream chambers. No more than 300 MeV of en- 
ergy could be present in any of the photon veto counters 
surrounding the vacuum decay region, the drift cham- 
bers, and the calorimeter. The energy deposited in the 
BA calorimeter was required to be less than 15 GcV to 
veto events in which an energetic photon escaped through 
one of the beam holes. 

Figure [2] shows the M^^ distribution for data out- 
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FIG. 1: Signal Monte Carlo events for the decay Kl 



7r°/x*e^ in the p? 



M^o^^ plane. 



All cuts except the sig- 



nal region cut have been made. The inner contour shows the 
signal region, and the outer contour indicates the control re- 
gion. 



side the signal and control regions, with all cuts ap- 
plied except the M^^ cut. This smooth distribution 
shows no peak at the 7r° mass. We therefore used 
the My-y sidebands above and below the 7r° mass re- 
gion (0.11 GeV/c2 < M^^ <0.132 GeV/c^ and 0.138 
GeV/c^ < M^^ < 0.16 GeV/c^), but inside the signal 
or control regions in L, to estimate the i^es backgrounds. 
The Ke3 background was thus estimated to be 0.56 ±0.23 
events in the signal region and 2.56 ±0.49 events in the 
control region. 

A second source of background was K^ — > vr^Tr^e^j/e 
(Kei), with a charged pion decay or punch through. A 
kinematic cut to reduce this background was defined by 
assuming a K^i decay and calculating the magnitude of 
the unseen neutrino's momentum in the K^ rest frame. 
For Ke4 decays, this quantity must be positive, while 
for signal decays it is usually negative. Requiring this 
variable to be negative removed most K^^ background. 
The remaining K^i contribution was determined from 
Monte Carlo simulation to be 0.10 ± 0.050 events in the 
signal region and 1.65 ± 0.20 events in the control region. 
Note that the Ke4 and K^s backgrounds must be added, 
since Ke4 decays do not contribute to the Ke3 sideband 
background estimate. 

Another possible source of background was Kl — + 
7r+7r^7r° decays. These decays could fake the signal if 
one charged pion decayed to a muon and the second was 
mistaken for an electron in the calorimeter and TRDs. 
However, due to the incorrect mass assignments, M^^o^^ 
reconstructed about 50 MeV/c^ below the true Kl mass, 
with no tail extending near the signal region. The ir/e 
rejection from both the calorimeter and the TRDs sup- 
press this background to a negligible level, as confirmed 
by both Monte Carlo simulation and Kl -^ 7r+7r~7r'' de- 
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FIG. 2: M™ distribution for Kl 



Tr'^/i e^ search data. 



for 



events outside the signal and control regions, with all cuts in 
place except the M^^, cut. The arrows show the regions used 
for the sideband background estimate. 



cays in data from a minimum-bias trigger. 

Other sources of background were considered but 
found to be negligible. We find an expected total back- 
ground of 0.66 ±0.23 events in the signal region and 4.21 
±0.53 events in the control region. 

The signal acceptance for Kl -^ n^n^e^ was deter- 
mined from Monte Carlo simulation to be 3.95% for the 
1999 data and 3.91% for the 1997 data. The total num- 
ber of Kl decays in the fiducial region was determined 
from the normalization mode to be (6.17 ± 0.31) x 10^^, 
and the single event sensitivity (SES) for the combined 
data set was (4.12 ± 0.21) x 10"" 0. 

When we opened the blind regions, we found events 
in the signal region and 5 events in the control region, 
consistent with background estimations. Figure [3] shows 
the P( — Mtt^c plane, with the surviving events shown 
as solid dots and the signal and control region shown as 
contours. 

The 90% confidence level (CL) upper limit was deter- 
mined for all modes in the following way. We stepped 
through a range of possible branching fractions, using a 
Monte Carlo simulation to produce a Poisson distribution 
at each value. The errors on the SES and backgrounds 
were taken into account by allowing these quantities to 
vary as Gaussian distributions with widths equal to their 
errors. The resulting Poisson distributions were then 
used to construct confidence bands, using the Feldman- 
Cousins prescription Q • From these confidence bands we 



determined BR{Kl ~* 7r°^±e=F) < 7.56 x 10"" at the 
90% CL. This result represents a factor of 82 irnprove- 
ment over the previous best limit for this mode. [8| 

We now consider the decay Kl — > 7r°7r°/i^e^. The 
addition of a second 7r° greatly reduces the backgrounds, 
so we were able to relax some cuts to improve the signal 
acceptance. Since Kl —> tt'^tt+tt" is not a background 
for this mode, we did not make a TRD requirement on 
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FIG. 3: Surviving events in the pt 

Kl — » Tv^fie^ search data. The signal and control regions 

are shown as the inner and outer solid contours. 



the electron track, and there was no cut on the number 
of partial track stubs. We ahowed up to two extra in- 
time hits in both the upstream and downstream drift 
chambers. 

Since we have two neutral pions in this decay, we can 
determine a neutral vertex independently of the charged 
vertex. We required that the difference between the neu- 
tral and charged vertices be less than 2.5 meters. In 
addition, we calculated an average vertex from the neu- 
tral and charged vertices, and recalculated M^^ using 
the average vertex. The resulting values were required to 
lie in the region 0.132 GeV/c^ < M^^ < 0.138 GeV/c^. 
Additionally, a kinematic cut on the square of the tt" 
momentum in the K^ rest frame was made on both tt^s. 

One important source of background for this mode was 
the decay Kl -^ tt^tt^tt^,. One electron could be mis- 
taken for a muon it was mismeasured in the calorimeter 
and if an accidental muon fired the appropriate muon ho- 
doscope paddles. To suppress this background, we made 
a loose cut on the TRD information for the muon track 
which rejected 85% of all electrons. This cut effectively 
eliminated K^ — > tt^tt^tt^ background. 



Other backgrounds arose from K,,^ or K^j^^ decays with 
four accidental photons. The M^^ sidebands could not be 
used in this case to estimate the background, since they 
did not have a smooth distribution. The background es- 
timate was obtained instead by the extrapolation of a 
linear fit to the log(£) distribution from outside the con- 
trol region into the signal and control regions. However, 
when all cuts were applied, there were not enough events 
remaining to make a reliable extrapolation. We there- 
fore defined three independent cut sets (kinematic cuts, 
particle ID cuts, and anti-accidental cuts). When we re- 
moved all three sets, we had sufiicient events to make 
an extrapolation into the signal region, as shown in fig- 




FIG. 4: The log(i) distribution for Kl -^ ■K°n°^i^e'^ search 
data. The three cuts sets as described in the text have 
been removed. A linear fit over the region -15<log(i)<5 
was extrapolated into the signal (log(i)>10) and control 
(5 <log(i)< 10) regions to estimate the background. The 
upper and lower dashed lines indicate the error bands used to 
assign a systematic error to the background estimate. 



urelH After the extrapolation, we apply the suppression 
factor associated with each cut set, as determined from 
the data. We verified from the data (by applying the cut 
sets in various combinations) that the three sets were in- 
deed independent, so that we could multiply the three 
separate suppression factors to get the final background 
estimate. The total number of background events was 
thus estimated to be 0.44 ± 0.23 in the signal region and 
0.43±0.17 in the control region. Due to the uncertainties 
in both the extrapolation in \og{L) and the suppression 
factors, we assign a systematic error on the background 
estimate by allowing the fit parameters to vary by 2.5 a 
from their central values. 

The signal acceptance was 2.04% for the 1999 data 
and 1.95% for the 1997 data. The total number of Kl 
decays was (6.36 ± 0.24) x 10". The SES for the com- 
bined data set was (7.88 ± 0.28) x 10"". When the 
blind regions were opened, we found no events in either 
the signal or control regions. We set the 90% CL limit 
BR{Kl -^ 7r°7r°^±e=F) < 1.64 x 10~i°, which is the first 
limit reported for this decay. 

The search for 7r° -^ /^^e^, tagged from Kl -^ 7r°7r°7r° 



is identical to the Kl 



7rO^V±e 



^ search with the 



additional requirement that M^e be in the tt^ mass re- 
gion. The background was estimated from both Kl -^ 
tt'^tt^tt^ Monte Carlo and from an extrapolation of the 
log(i) distribution into the signal region as was done 
for Kl -^ TT^TT^/i^e^. The two methods gave con- 
sistent results, yielding a background estimate of 0.03 
±0.015 events in the signal region and an identical value 
in the control region. The fiux for this mode was de- 
termined from {Kl decays) x3 x BR{Kl -^ 7r°7r°7r°), 
yielding a SES of (1.48 ± 0.059) x lO^^". When the 



blind regions were opened, we found no events in ei- 
ther the signal or control regions. We set the 90% CL 
limit BR{tt° -> fi^e^) < 3.59 x 10-i°. Our limit on 
7r° — > jjL^e^ is equally sensitive to both c harg e modes, 
while the previous best limits were not [9|,[l0|. Assum- 
ing equal contributions from both charge combinations, 
our result is about a factor of two better than the pre- 
vious best limit on tt^ — > /i+e^ and about a factor of 10 
greater than the previous best limit on vr*' — )■ /i^e+. 

Although no evidence for these flavor-violating modes 
has been found, the pursuit should not dropped. Given 
that we find negligible backgrounds, our techniques could 
clearly be extended to higher intensity neutral kaon 
beams. 
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